Aggregation of related media content

ABSTRACT

Systems and methods for media aggregation are disclosed herein. The system includes a media system that can transform media items into one aggregated media item. A synchronization component synchronizes media items with respect to time. The synchronized media items can be analyzed and transformed into an aggregated media item for storage and/or display. In one implementation, the aggregated media item is capable of being displayed in multiple ways to create an enhanced and customizable viewing and/or listening experience.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/842,506, filed Sep. 1, 2015, which is a continuation of U.S. patentapplication Ser. No. 13/361,778, filed Jan. 30, 2012, each of which ishereby incorporated by reference herein in its entirety.

TECHNICAL FIELD

This disclosure relates to aggregation of related media content, andmore particularly to user controllable aggregation of media content fora particular event.

BACKGROUND

The proliferation, advancement, and affordability of image capturingdevices such as smart phones, digital cameras, and other electronicdevices has made media capturing easier and more available to thegeneral public than ever before. Sharing of videos captured byindividuals attending a particular sporting event, wedding, musicconcert or birthday party, etc. by way of a social media website hasbecome commonplace. Videos of a same event are often uploaded bydisparate people that attended the event. For example, querying anexisting media sharing website for a video of an event, such as “rockstar's concert in Berlin,” can result in dozens if not hundreds of videofiles of that event. Respective videos of a common event are oftenrecorded at different angles of view, different distance from the targetof the video, different locations with respect to acoustic and/orlighting quality, different device capabilities (e.g., image capturequality, encoding, audio sensors, processing capabilities, etc.), andother differing aspects that manifest in the recorded video.

SUMMARY

The following presents a simplified summary of the specification inorder to provide a basic understanding of some aspects of thespecification. This summary is not an extensive overview of thespecification. It is intended to neither identify key or criticalelements of the specification nor delineate the scope of any particularimplementations of the specification, or any scope of the claims. Itspurpose is to present some concepts of the specification in a simplifiedform as a prelude to the more detailed description that is presented inthis disclosure.

Systems and methods disclosed herein relate to aggregation of mediacontent. In one non-limiting exemplary implementation, an identificationcomponent receives a media item and associates the media item with mediaitems of the same event. A synchronization component synchronizes themedia items based on a time correlation. Optionally, an analyzercomponent can be employed to analyze and rank the media items orsegments of the media items, based on at least one criterion, e.g.,video quality. An aggregation component then aggregates the synchronizedand ranked portions of media items for aggregated playback. Optionally,an editor component can allow a user to modify and/or customizeaggregation.

In another non-limiting example, a set video files are associated witheach other. The video files, or portions of the files, are thensynchronized based on time, then analyzed and ranked based on a metric(e.g., quality and/or user preference). The video files are aggregatedaccording to rank. The aggregated files can be viewed by a user(s). Theuser(s) can seamlessly change between video files, audio files, and/orview multiple video or audio files at the same time.

The following description and the drawings set forth certainillustrative aspects of the specification. These aspects are indicative,however, of but a few of the various ways in which the principles of thespecification may be employed. Other advantages and novel features ofthe specification will become apparent from the following detaileddescription of the specification when considered in conjunction with thedrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Numerous aspects, implementations, and advantages of the presentinvention will be apparent upon consideration of the following detaileddescription, taken in conjunction with the accompanying drawings, inwhich like reference characters refer to like parts throughout, and inwhich:

FIG. 1 illustrates a high-level functional block diagram of an examplemedia system in accordance with various aspects of this disclosure;

FIG. 2 illustrates an exemplary synchronization of media items inaccordance with various aspects of this disclosure;

FIG. 3 illustrates a high-level functional block diagram of an examplemedia system including an editing component in accordance with variousaspects of this disclosure;

FIG. 4 illustrates a high-level functional block diagram of an examplemedia system including a view component in accordance with variousaspects of this disclosure;

FIG. 5 illustrates an example diagram of a real world event inaccordance with various aspects of this disclosure;

FIG. 6 illustrates an example methodology for aggregating media items inaccordance with various aspects of this disclosure;

FIG. 7 illustrates an example methodology for aggregating media itemsaccording to user input in accordance with various aspects of thisdisclosure;

FIG. 8 illustrates an example methodology for aggregating media itemsand displaying media items in panoramic format in accordance withvarious aspects of this disclosure;

FIG. 9 illustrates an example schematic block diagram of a computingenvironment in accordance with various aspects of this disclosure; and

FIG. 10 illustrates an example block diagram of a computer operable toexecute various aspects of this disclosure.

DETAILED DESCRIPTION

Various aspects or features of this disclosure are described withreference to the drawings, wherein like reference numerals are used torefer to like elements throughout. In this specification, numerousspecific details are set forth in order to provide a thoroughunderstanding of this disclosure. It should be understood, however, thatcertain aspects of disclosure may be practiced without these specificdetails, or with other methods, components, materials, etc. In otherinstances, well-known structures and devices are shown in block diagramform to facilitate describing this disclosure.

Systems and methods disclosed herein relate to aggregation of mediaitems. In one implementation, a media system is employed to aggregate aplurality of videos. Users capture and upload multiple video files of anevent, such as for example a wedding, a school play, a children'ssporting event, or the like (employing any of a variety of suitablevideo capture devices, e.g., video cameras, cell phones, tablets, smartphones, etc.). Respective video files are uploaded to a site forpublication so that the user and others (e.g., friends, family membersand other contacts) can access the video. The video files are uploadedin any suitable manner, e.g., directly from a recording device, using apersonal computer, smart phone, or tablet.

In an embodiment, the video file(s) are stored on one or more servers.Each server may include various components that perform one or morerespective functions such as analyzing the videos, acquiring features ofthe videos (e.g., determining the event that was recorded, a time ofrecording, the video's author, the type of recording device, the video'sformat, the event location, individuals within the video, angle ofrecording, the video's quality, the quality of associated audio,lighting, etc.), classifying the videos, associating the videos withsimilar videos (e.g., videos of the same event), and providing foraggregating subsets of the videos.

In one implementation, a component stored in a server determines thatcertain videos are related based on, for example, metadata or byanalyses of the content of the videos. Metadata can include informationrelated to time of capture, location of capture, and/or tags, forexample. In one aspect, time and location information can be providedvia a global positioning system (GPS) or communication network accesspoints (e.g., internet connection access points). Time and locationinformation associated with respective videos can determine arelationship between of the respective videos. Tags can be associatedwith each video by users or by automatic tagging techniques. Taggingrefers to associating a key word, set of words or phrase with the video,wherein the associated key word(s) or phrases are referred to as atag(s). For example, a user can tag a captured vide with “rock concert”“New York” “2000” “Rock Star”.

In one embodiment, a subset of related videos is subsequentlysynchronized. The synchronization can be performed according to a commontime line. For example, the time line can be determined through audiorecognition, pattern recognition, captured time (e.g., time stamp), or acombination thereof. The videos can be analyzed in connection withvarious metrics or criteria (e.g., video quality, audio quality, angleof recording, level of noise, lighting, frequency of appearance ofcertain people or objects, etc.). For example, with respect to a set ofvideos associated with a particular school concert, a user may wish tofocus on a particular child (the parent's own child) in the concert andinput such criteria. Thus, a video can be aligned according to userrelevance based in part on an affinity of a user to an individual oritem viewable in the subset of the plurality of media items. Analyses ofthe set videos can determine (e.g., using facial recognition) image sizeand frequency of appearance of the child's face in respective videos ofthe set. In one implementation, videos (or segments thereof) within theset can be ranked based on a specified metric(s) (e.g., image size, andfrequency of appearance of the child).

Subsets of portions of the respective videos can be aggregated togenerate one or more files that emphasize desired aspects of theanalyzed videos. The aggregated file(s) can display unique video in avariety of manners such as a single composite video, a mosaic of videos,a set of stitched videos for panoramic and/or 360 degree display.Furthermore, in one implementation, a user can choose to switch betweenvideos for display. With reference to the above school concert example,the aggregated file can include seamless switching between a subset ofthe plurality of videos as a function of respective rank.

In an implementation, sound associated with each video can be combinedor selectively chosen for playback. For example, analysis of audioinformation can determine highest quality audio associated withrespective videos. Then, the selected audio can be played based on rankand/or combined with other audio. Audio of one video can be combinedwith an image portion of another video. Respective aggregated files cancontain surround sound qualities through aggregation of multiple audiofiles.

Media items can be synchronized, analyzed, and aggregated according toaudio quality, video quality, audio characteristics (e.g., focusing onbass, treble, amplitude, pitch, dB level, etc.), video features, or thelike.

Referring now to FIG. 1, there is illustrated a non-limiting exemplaryimplementation of a media system 100 that provides media aggregation inaccordance with various aspects of this disclosure. The system 100provides for aggregation of media items that are captured, uploadedand/or viewed. The system 100 receives media item(s) 114 (e.g., audioand or video information) that are respectively captured and uploaded tothe system 100 by a set of users. Respective users can capture andupload the media item(s) 114 via any suitable electronic device, such asfor example a camera, smart phone, video camera, tablet computer, laptopcomputer, personal computer, cell phone and the like, via acommunication framework, e.g., the Internet or cellular network. Themedia item(s) 114 can be provided to an identification component 110 atconcurrent, overlapping or disparate times.

In FIG. 1, the system 100 includes a computer processing unit 104capable of executing various components stored in a computer readablememory 106, such as an identification component 110, a synchronizationcomponent 120, an analyzer component 130, and an aggregation component140. Memory 106 can store media item(s) 114. Identification component110 associates a subset of the media items that pertain to a commonevent (e.g., a concert, a play, a sporting event, a speech) and/or acommon person, object, subject, etc.). Identification component 110 canidentify the common event through extrinsic information or analysesassociated with respective media items (e.g., image or audiorecognition, metadata comparison, tagging information, authors,location, time, content, etc.). In one example, metadata is generatedautomatically, such as via a global positioning system (GPS), orwireless location triangulation. In addition, time and locationinformation can be generated by communication framework access points orrouting information. For example, users can upload respective mediaitems via a wireless router in a school auditorium and mediaidentification component 110 can associate the respective media itemswith that location.

Synchronization component 120 synchronizes associated sets and/orsubsets of media items. The associated media items are synchronized as afunction of common characteristics (e.g., sound recognition, motionrecognition, pattern recognition, time stamps, etc.). In one aspect, themedia information synchronization is performed with respect to time.Synchronization component 120 can identify a common time line bycomparing like characteristics in the set and/or subset of mediaitem(s), metadata such as time stamps, or a combination thereof. In oneexample, motion and/or pattern recognition identifies common movementsof a pattern in a plurality of video files to synchronize a set of mediaitems. In another aspect, each frame of a video file can be synchronizedwith overlapping frames of other video items. In another example, keyframes are selected for synchronization. In the event that audioinformation is contained in video files, synchronization can rely atleast in part on correlation of the audio information. In an aspect,comparison of audio signals can be utilized to create a common time linefor synchronization of associated video signals.

Analyzer component 130 analyzes a set of media items based on any of avariety of suitable metrics (e.g., quality of video, quality of audio,frequency of appearance of an individual or object, and angle of videoor audio) that facilitate identifying portions of respective media itemswith desirable attributes. In one embodiment, analyzer component 130groups a set of media items associated with a common event into one ormore subsets of media items. Respective subsets of media items can begrouped based on any of a variety of suitable metrics.

In one example, analyzer component 130 ranks media items and groups themedia items into respective subsets based on the rank (e.g., ranking andgrouping as a function of video quality). In one aspect, analyzercomponent 130 segments media items into segments comprised of frames andanalyzer component 130 analyzes each segment. Video quality analysis caninclude comparison of distortion, compression rates, variance inluminance and chrominance channels, comparison of noise, comparison ofpattern frequency appearances and comparison of size. Video quality maybe evaluated via video quality assessment (VQA) algorithms andtechniques. Such techniques may include peak signal to noise ratio(PSNR), structural SIMilatary (SSIM), Multi-scale SSIM (MS-SSIM), SpeedSSIM, and/or Visual Signal to Noise Ratio (VSNR).

Additionally and/or alternatively, video quality can be determinedsubjectively through user analysis. In one example, users can associatea quality with a video. The quality may be a level on a subjective scalesuch as number of stars or a number out of ten. In another example,video quality is subjectively determined based on multiple user input.Multiple user input may be an average ranking, a cumulative ranking(e.g., users may increase or decrease a quality by one), and/or acombination of the above.

Audio component quality associated with respective media items can vary.Audio component quality variation may be a result of several factorssuch as for example electronic equipment used in capturing audio,compression and decompression, orientation and distance from sourceaudio, and/or background noise.

In one aspect, audio component quality can be analyzed via audio qualityevaluation techniques. In one example, audio quality evaluation can bebased on perceptual based audio quality methods (e.g., perceptualevaluation of audio quality (PEAQ), perceptual evaluation of speechquality (PESQ)), non-perceptual based audio quality methods (e.g., totalharmonic distortion (THD), signal to noise ratio (SNR)), noise ratio(NR), spectral distortion (e.g., Bark spectral distortion), and/orcomparison of parameters (e.g. loudness, amplitude modulation, adaption,masking parameters, linear distortion, bandwidth, and modulardifference). In another example, at least one of psychoacousticevaluation models or cognitive evaluation models can be employed toevaluate audio component quality.

In another example, audio quality can be determined through user'ssubjective analysis. For example, one or more users can associate aquality with an audio aspect of a media item.

In one implementation, analyzer component 130 uses pattern recognitiontechniques and/or systems to determine a rank associated with each mediaitem of a set of media items 114. The analyzer component 130 candetermine a frequency rate of a pattern (e.g., how often a patternappears) in media items of the plurality of media items 114. Analyzercomponent 130 can associate a higher rank with media items of theplurality of media items 114 with higher frequencies.

Additionally or alternatively, pattern recognition techniques canidentify a size of a pattern and/or identify partial patterns, e.g.,only a portion of a total pattern is in an image frame. Moreover, aranking can be determined based on the overall appearance and frequencyof an identified pattern.

In another example, analyzer component 130 associates a rank with eachmedia item of a set of media items according to a metric. For example,analyzer component 130 may analyze video quality of each media item ofthe set of media items 114 and analyzer component 130 may associate arank with media items according to respective video quality. In oneaspect, analyzer component 130 determines a respective locationassociated with media items based on a field. A location can bedetermined via user input or via image pattern recognition and/or motionrecognition technology. Additionally, analyzer component may associate arank with each media item of a set of media items based on the relativelocation.

In one aspect, each media item may not completely overlap with respectto time. Analyzer component 130 can divide media items into segments orclips. Segments of respective media items can be ranked with segments ofother media items which temporally coincide.

In another aspect, analyzer component 130 can separate media items intoseparate components (e.g., audio component and video component).Respective components of media items can be analyzed. For example, audiocomponents and video components of respective media items of a set ofmedia items may be analyzed and ranked. In one example, correspondingaudio and video components associated with the same media item can begiven a rank independent of each other.

Aggregation component 140 transforms a set of media items into acomposite aggregated media item. In one aspect, the composite aggregatedmedia item can contain a set of synchronized and analyzed media items.In another example, a plurality of videos can relate to a common timesuch that the plurality of videos can be displayed simultaneously.

In on example, the composite aggregated media item can contain a subsetof synchronized and analyzed media items such that only a select numberof media items are aggregated. The subset can be determined based onanalyzed metrics of the set of media items.

In one example, the composite media item can be a mosaic media item forstorage and/or output. A mosaic aggregated media item may include avideo array containing video segments associated with a plurality ofmedia items such that, during playback, a plurality of video segments isconcurrently displayed to create a mosaic display. In another aspect, amosaic aggregated media item may include an audio array containing audiosegments associated with a plurality of media items such that, duringplayback, a plurality of audio components is concurrently played tocreate surround sound qualities. Optionally, the audio signals arealtered for appropriate playback (e.g., normalized and/or noisesuppression).

In another example, a particular video segment of the mosaic media itemcan be focused on during playback while other video segments are blurredor partially displayed. Focus can be determined based on ranking, and/oruser preference.

In another example, aggregation component 140 can create compositeaggregated media item that is rotatable. For example, media items can bestitched to create a panoramic media item. The panoramic media item cancomprise media items stitched according to relative location. Thepanoramic media item can be rotatable such that one or more videosegments can be displayed and then one or more different video segmentscan be displayed. Display can change based on ranking, user input,positional information, and/or periodically (e.g., change every fiveseconds).

In one aspect, aggregation component 140 can implement techniques tocalibrate video channel levels (e.g., brightness, color, gamma) of eachmedia item of the plurality of media item(s) 114 such that video itemscontain similar levels. For example, media items can each contain adistinct level of brightness. An ideal brightness can be selected,(e.g., brightness can be averaged, highest level of brightness, a rangeof brightness) and each media item can be adjusted, amplified orde-amplified, to an appropriate level.

In another example, aggregation component 140 can implement techniquesto calibrate audio levels of each media item of the plurality of mediaitem(s) 114 such that audio files contain similar levels. For example,media items can contain various volumes of audio. An ideal volume can beselected, (e.g., volumes can be averaged, loudest volume, a range ofvolumes) and respective audio components are adjusted, amplified and/orde-amplified to an appropriate level. Calibration can be automated, suchas implemented by a processor running computer executable instructions,can be manual, or can be implemented as a combination of automated andmanual calibration.

Turning to FIG. 2, with reference to FIG. 1, a graphical depiction 200of media items synchronized on a common timeline 202 is provided. InFIG. 2, four different user captured video items (media items A, B, C,and D (205, 215, 220, 225)) are shown. The media items pertain to acommon event. Each respective video item can be taken over same,different, or overlapping time periods. Each respective video item maybe taken using different types of devices, e.g., different consumerelectronic devices. For example, video items may be taken using a mobilephone by one user, a point-and-shoot digital camera by another user, anda different portable device by another user. While only four differentmedia items are shown for illustrative purposes, a different numbermedia items could be depicted instead.

In one example, analyzer component 130 selects media items A, B, C, andD (205, 215, 220, 225) as a subset of media items from a set of mediaitems 114 identified by identification component 110. The set of mediaitems can represent media items with a desired feature (e.g., each mediaitem contains a common image pattern or is a desired quality).

Synchronization component 120 can identify the common feature(s) 210 ofmedia items A, B and C (205, 215, 220). In one aspect, the commonfeature(s) may be recognized via image recognition, motion recognition,and/or audio recognition. Additional feature(s) can be identifiedbetween at least two media items of the set of media items A-D. In oneaspect, synchronization component 120 can synchronize media items one ormore times. In one example, media items A, B, C, and D (205, 215, 220,225) may be synchronized according to identified key frames, frame byframe, randomly, and/or periodically.

Synchronization component 120 identifies or defines a common timeline202. The common timeline 202 can have associated start and end times foreach media item A, B, C, and D (205, 215, 220, 225). FIG. 2 depictsmedia items with the following time associations: media item A (205)begins at a time T0 and lasts until T6; media item B (215) begins attime T2 and lasts until T7; media item C (220) begins at T1 and lastsuntil T5; and media item D (225) begins at T3 and lasts until T4.

In one aspect, analyzer component 130 associates a rank with each mediaitem of the set of media items A, B, C, and D (205, 215, 220, 225). Inanother aspect, analyzer component splits each media item of the set ofmedia items A, B, C, and D (205, 215, 220, 225) into one or more clips.Clips can be various lengths. In one example, analyzer component 130determines clip lengths according to start and end times of each mediaitem of the set of media items A, B, C, and D (205, 215, 220, 225). Inthis aspect, media item A (205) is divided into six clips (T0 to T1, T1to T2, T2 to T3, T3 to T4, T4 to T5, T5 to T6) and the remaining mediaitems are similarly divided.

Analyzer component 130 can rank respective clips of media items on atemporal basis. For example, analyzer component 130 can rank respectiveclips of media item A (205) and C (220) associated with the period T1 toT2. In addition, analyzer component can rank respective clips of mediaitem A (205), B (215) and C (220) associated with period T2 to T3. In animplementation, analyzer component can store ranks of respective clipsin an array.

Aggregation component 140 can aggregate the set of media items A, B, C,and D (205, 215, 220, 225) into one or more aggregated media items. Forexample, aggregation component 140 can aggregate media items A, B, C,and D (205, 215, 220, 225) as a function of rank. In one example, mediaitems with a higher rank are displayed or identified as a focal point.

Referring to FIG. 3, there is illustrated a non-limiting exemplaryimplementation of media system 300 that provides media item aggregationaccording to an aspect of this disclosure. Media system 300 includes acomputer processing unit 304 capable of executing various componentsstored in a computer readable memory 306, such as an identificationcomponent 310, a synchronization component 320, an analyzer 330, anaggregation component 340, and an editing component 350. Identificationcomponent 310 collects media items and associates media items of commonevents. Synchronization component 320 synchronizes media items withrespect to time. Analyzer component 330 analyzes aspects of media itemsand can rank media items, or clips of media items. Aggregation component340 aggregates the analyzed and synchronized media items into acomposite media item.

In one implementation, editing component 350 modifies analysis andranking by analyzer component 330 and aggregation by aggregationcomponent 340, e.g., based on user input 360. In one example, user input360 can customize analysis to place a higher importance on frequency ofan image pattern, e.g., place higher importance on recognition of aparticular face in a set of media items. User input 360 can beinformation input by at least one user and system 300 can receive and/orsave user input 360.

In another example, editing component 350 can enable modification ofranking and aggregation according to arbitrary user input 360. Forexample, a user may prefer a particular view and/or may prefertransition or panning to a different view based on personal preferenceand not a measurable metric.

Turning now to FIG. 4, there is illustrated a non-limiting exemplaryimplementation of a media system 400 that provides media aggregation,according to an aspect of this disclosure. Media system 400 includes acomputer processing unit 404 capable of executing various componentsstored in a computer readable memory 406, such as an identificationcomponent 402, a synchronization component 410, an analyzer 430, anaggregation component 440, a view component 450, and an output component460. Identification component 402 receives media items 414 and associatemedia items 414 into sets of media items pertaining to respective commonevents. Synchronization component 410 synchronizes media items withrespect to time. Analyzer component 430 analyzes and ranks mediaaccording to at least one metric. Aggregation component 440 transforms aset of media items to at least one aggregated media item.

In FIG. 4, output component 460 is capable of outputting output mediaitems 470 (e.g., a composite aggregated media item). In one aspect,outputting a media item can include uploading, storing in memory,broadcasting, displaying and/or streaming. In an example, outputcomponent 460 outputs a mosaic media item or a panoramic media item.

Referring now to FIG. 5 with reference to FIG. 4, there is illustrated adiagram of an event in accordance with various aspects of thisdisclosure. In FIG. 5, the event is a school concert where there is astage 502 and a student performer 504 on the stage 502. A plurality ofelectronic devices (510, 512, 514, 516) captures media items of theevent. Each media item captured by the electronic devices (510, 512,514, 516) has an associated field of view (520, 522, 524, 526,respectively). In one example, media items captured by the plurality ofelectronic devices (510, 512, 514, 516) vary in audio quality, videoquality, angle, and capture content (e.g., focus on different areas ofstage 502). In use, each electronic capture device may be owned by adifferent user (e.g., a different parent of students in the schoolconcert) and each user may upload the media item captured by his/herdevice to a user generated content (UCG) hosting site. In oneimplementation, view component 450 provides for panning between fieldsof view of respective media items. For example, view component 450 mayallow for output media information 460 to display the field of view 520and enable a user to pan to view 526 by transitioning to view 522, 524and then 526. In one implementation, the output component 460 displaysmore than one segment of a set of media items simultaneously.

Referring now to FIGS. 6-8, there are illustrated methodologies and/orflow diagrams in accordance with the disclosed subject matter. Forsimplicity of explanation, the methodologies are depicted and describedas a series of acts. However, acts in accordance with this disclosurecan occur in various orders and/or concurrently, and with other acts notpresented and described herein. For example, synchronization can occurbefore analysis and/or multiple synchronizations can occur. Furthermore,not all illustrated acts may be required to implement the methodologiesin accordance with the disclosed subject matter. In addition, thoseskilled in the art will understand and appreciate that the methodologiescould alternatively be represented as a series of interrelated statesvia a state diagram or events. Additionally, it should be furtherappreciated that the methodologies disclosed hereinafter and throughoutthis specification are capable of being stored on an article ofmanufacture to facilitate transporting and transferring suchmethodologies to computers. The term article of manufacture, as usedherein, is intended to encompass a computer program accessible from anycomputer readable device or storage medium.

With reference to FIG. 6, there is illustrated a methodology 600 foraggregating media information, according to an aspect of thisdisclosure. As an example, various media applications, such as, but notlimited to, media capturing systems, social networking systems, and usergenerated content (UGC) systems, can use methodology 600. Specifically,methodology 600 transforms multiple media items into a single aggregatedmedia item.

Media items can be collected (e.g., by an identification component) at602. For example, media items can be stored in a computer readablememory (e.g., memory 104). The memory can be volatile or nonvolatilememory, for example. At 604, media items are associated with other mediaitems (e.g., by an identification component). For example, media itemspertaining to a common event can be associated by a media system.

At 606, media items are analyzed (e.g., by an analyzer component).Analyzing media items may include media quality determination, image oraudio signal recognition, ranking of media items, division of mediaitems into segments, and configuration of media levels (e.g., soundconfiguration, image configuration) for example.

At 608, media items are synchronized (by a synchronization component).In one aspect, media items are synchronized with respect to time.Synchronization may be based on identifiable aspects of mediainformation such as common audio or video characteristics. Additionallyor alternatively, synchronization may be based on metadata such as timestamps.

At 610, media items can be ranked (by an analyzer component). Rankingcan comprise associating a unique rank with each media item according toanalyzed metrics and/or features. In one aspect, respective media itemclips corresponding to identical time periods are ranked relative toeach other. Ranking can rely on comparison of associated audio quality,video quality, frequency rate of an identified patter, and/or field ofview.

Turning to 612, media items are transformed into one or more aggregatedmedia item (e.g., by an aggregation component). The aggregated mediaitem can comprise multiple video items capable of being simultaneouslydisplayed in a mosaic, panoramic and/or 360 degree display.

Referring to FIG. 7 there illustrated is a methodology 700 for userspecific media item aggregation according to an aspect of thisdisclosure. As an example, methodology 700 can be utilized in variousweb applications, media capturing systems, media displaying systems,computing devices, cellular phones, etc. Specifically, methodology 700enables utilization of user defined criteria to provide for aggregatemedia files of a common event which most interest a user.

Initially, media items can be captured or can be contained withinmemory. Media items may be video with accompanying audio. At 702, a usercan define criteria for aggregation (e.g., via an editor component 350).Criteria for aggregation can be related to identifiable characteristicsof media items such as frequency of an image in frames, video quality,audio quality, image characteristics, video characteristics (motion,brightness, color), and/or sound characteristics (e.g., bass, treble,noise), for example.

At 704, a set of media items is synchronized with respect to time (e.g.,by a synchronization component 320) At 706, media items are analyzedaccording to the user defined criteria (e.g., by an analyzer component330). Media items can be grouped into sets according to the analyzedcriteria. For example, media items that contain a common image patterncan be grouped into a set (e.g., media items containing a recognizedface can be grouped into a set).

At 708, each media item of a set of media items are ranked according toa user relevance determined by the user defined criteria (e.g., by aanalyzer component 330). For example, a user may select video quality asthe most important attribute for ranking. Thus, media items with higherassociated video quality will have a higher rank. At 710 sets of mediaitems are transformed into one or more aggregated media items (e.g., byan aggregation component 340). In one aspect, the aggregated media itemcontains each media item of the set of media items. In another aspect,the aggregated media item contains a subset of the set of media items.

FIG. 8 illustrates an example methodology 800 for media item aggregationand display in accordance with an aspect of this disclosure. Initially aplurality of media items is analyzed, synchronized, and ranked (e.g., byan analyzer component). At 802, an aggregation component, for example,can aggregate media items with respect to a rank. In another example,media items can be aggregated with respect to an associated field ofview as compared to other media items to generate a composite mediaitem.

At 804, a composite media item is output (e.g., by an output component).Outputting the composite media items can include saving in a computerreadable memory, streaming, uploading, downloading, broadcasting and/ordisplaying the composite media item (e.g., by an output component). Forexample, the outputted composite media items may be displayed, e.g., viaa communications framework (e.g., the internet, cellular networks,Ethernet and/or wireless networks) in communication with a displayscreen.

At 806, the composite media item can switch between views. For example,an editing component can receive user input to pan through views of thecomposite media item. A view component can switch the view of thecomposite media item based on the received input and an output componentcan display the changed view.

The systems and processes described below can be embodied withinhardware, such as a single integrated circuit (IC) chip, multiple ICs,an application specific integrated circuit (ASIC), or the like. Further,the order in which some or all of the process blocks appear in eachprocess should not be deemed limiting. Rather, it should be understoodthat some of the process blocks can be executed in a variety of ordersthat are not all of which may be explicitly illustrated herein.

With reference to FIG. 9, a suitable environment 900 for implementingvarious aspects of the claimed subject matter includes a computer 902.The computer 902 includes a processing unit 904, a system memory 906, acodec 905, and a system bus 908. The system bus 908 couples systemcomponents including, but not limited to, the system memory 906 to theprocessing unit 904. The processing unit 904 can be any of variousavailable processors. Dual microprocessors and other multiprocessorarchitectures also can be employed as the processing unit 904.

The system bus 908 can be any of several types of bus structure(s)including the memory bus or memory controller, a peripheral bus orexternal bus, and/or a local bus using any variety of available busarchitectures including, but not limited to, Industrial StandardArchitecture (ISA), Micro-Channel Architecture (MSA), Extended ISA(EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB),Peripheral Component Interconnect (PCI), Card Bus, Universal Serial Bus(USB), Advanced Graphics Port (AGP), Personal Computer Memory CardInternational Association bus (PCMCIA), Firewire (IEEE 1394), and SmallComputer Systems Interface (SCSI).

The system memory 906 includes volatile memory 910 and non-volatilememory 912. The basic input/output system (BIOS), containing the basicroutines to transfer information between elements within the computer902, such as during start-up, is stored in non-volatile memory 912. Byway of illustration, and not limitation, non-volatile memory 912 caninclude read only memory (ROM), programmable ROM (PROM), electricallyprogrammable ROM (EPROM), electrically erasable programmable ROM(EEPROM), or flash memory. Volatile memory 910 includes random accessmemory (RAM), which acts as external cache memory. By way ofillustration and not limitation, RAM is available in many forms such asstatic RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), doubledata rate SDRAM (DDR SDRAM), and enhanced SDRAM (ESDRAM).

Computer 902 may also include removable/non-removable,volatile/non-volatile computer storage media. FIG. 9 illustrates, forexample, a disk storage 914. Disk storage 914 includes, but is notlimited to, devices like a magnetic disk drive, solid state disk (SSD)floppy disk drive, tape drive, Zip drive, LS-100 drive, flash memorycard, or memory stick. In addition, disk storage 914 can include storagemedia separately or in combination with other storage media including,but not limited to, an optical disk drive such as a compact disk ROMdevice (CD-ROM), CD recordable drive (CD-R Drive), CD rewritable drive(CD-RW Drive) or a digital versatile disk ROM drive (DVD-ROM). Tofacilitate connection of the disk storage devices 914 to the system bus908, a removable or non-removable interface is typically used, such asinterface 916.

It is to be appreciated that FIG. 9 describes software that acts as anintermediary between users and the basic computer resources described inthe suitable operating environment 900. Such software includes anoperating system 918. Operating system 918, which can be stored on diskstorage 914, acts to control and allocate resources of the computersystem 902. Applications 920 take advantage of the management ofresources by operating system 918 through program modules 924, andprogram data 926, such as the boot/shutdown transaction table and thelike, stored either in system memory 906 or on disk storage 914. It isto be appreciated that the claimed subject matter can be implementedwith various operating systems or combinations of operating systems.

A user enters commands or information into the computer 902 throughinput device(s) 928. Input devices 928 include, but are not limited to,a pointing device such as a mouse, trackball, stylus, touch pad,keyboard, microphone, joystick, game pad, satellite dish, scanner, TVtuner card, digital camera, digital video camera, web camera, and thelike. These and other input devices connect to the processing unit 904through the system bus 908 via interface port(s) 930. Interface port(s)930 include, for example, a serial port, a parallel port, a game port,and a universal serial bus (USB). Output device(s) 936 use some of thesame type of ports as input device(s) 928. Thus, for example, a USB portmay be used to provide input to computer 902, and to output informationfrom computer 902 to an output device 936. Output adapter 934 isprovided to illustrate that there are some output devices 936 likemonitors, speakers, and printers, among other output devices 936, whichrequire special adapters. The output adapters 934 include, by way ofillustration and not limitation, video and sound cards that provide ameans of connection between the output device 936 and the system bus908. It should be noted that other devices and/or systems of devicesprovide both input and output capabilities such as remote computer(s)938.

Computer 902 can operate in a networked environment using logicalconnections to one or more remote computers, such as remote computer(s)938. The remote computer(s) 938 can be a personal computer, a server, arouter, a network PC, a workstation, a microprocessor based appliance, apeer device, a smart phone, a tablet, or other network node, andtypically includes many of the elements described relative to computer902. For purposes of brevity, only a memory storage device 940 isillustrated with remote computer(s) 938. Remote computer(s) 938 islogically connected to computer 902 through a network interface 942 andthen connected via communication connection(s) 944. Network interface942 encompasses wire and/or wireless communication networks such aslocal-area networks (LAN), wide-area networks (WAN), and cellularnetworks. LAN technologies include Fiber Distributed Data Interface(FDDI), Copper Distributed Data Interface (CDDI), Ethernet, Token Ringand the like. WAN technologies include, but are not limited to,point-to-point links, circuit switching networks like IntegratedServices Digital Networks (ISDN) and variations thereon, packetswitching networks, and Digital Subscriber Lines (DSL).

Communication connection(s) 944 refers to the hardware/software employedto connect the network interface 942 to the bus 908. While communicationconnection 944 is shown for illustrative clarity inside computer 902, itcan also be external to computer 902. The hardware/software necessaryfor connection to the network interface 942 includes, for exemplarypurposes only, internal and external technologies such as, modemsincluding regular telephone grade modems, cable modems and DSL modems,ISDN adapters, wired and wireless Ethernet cards, hubs, and routers.

Referring now to FIG. 10, there is illustrated a schematic block diagramof a computing environment 1000. The system 1000 includes one or moreclient(s) 1002, (e.g., computers, smart phones, tablets, cameras,PDA's). The client(s) 1002 can be hardware and/or software (e.g.,threads, processes, computing devices). The client(s) 1002 can housecookie(s) and/or associated contextual information by employing thespecification, for example.

The system 1000 also includes one or more server(s) 1004. The server(s)1004 can also be hardware or hardware in combination with software(e.g., threads, processes, computing devices). The servers 1004 canhouse threads to perform transformations by employing aspects of thisdisclosure (e.g., analyzing videos, aggregating videos, and/or storingvideo information). One possible communication between a client 1002 anda server 1004 can be in the form of a data packet adapted to betransmitted between two or more computer processes wherein data packetsmay include coded media items. The data packet can include a cookieand/or associated contextual information, for example. The system 1000includes a communication framework 1006 (e.g., a global communicationnetwork such as the Internet) that can be employed to facilitatecommunications between the client(s) 1002 and the server(s) 1004.

Communications can be facilitated via a wired (including optical fiber)and/or wireless technology. The client(s) 1002 are operatively connectedto one or more client data store(s) 1008 that can be employed to storeinformation local to the client(s) 1002 (e.g., cookie(s) and/orassociated contextual information). Similarly, the server(s) 1004 areoperatively connected to one or more server data store(s) 1010 that canbe employed to store information local to the servers 1004.

In one implementation, a client 1002 can transfer an encoded file, inaccordance with the disclosed subject matter, to server 1004. Server1004 can store the file, decode the file, or transmit the file toanother client 1002. It is to be appreciated, that a client 1002 canalso transfer uncompressed file to a server 1004 and server 1004 cancompress the file in accordance with the disclosed subject matter.Likewise, server 1004 can encode video information and transmit theinformation via communication framework 1006 to one or more clients1002.

The illustrated aspects of the disclosure may also be practiced indistributed computing environments where certain tasks are performed byremote processing devices that are linked through a communicationsnetwork. In a distributed computing environment, program modules can belocated in both local and remote memory storage devices.

Moreover, it is to be appreciated that various components describedherein can include electrical circuit(s) that can include components andcircuitry elements of suitable value in order to implement theimplementations of this innovation(s). Furthermore, it can beappreciated that many of the various components can be implemented onone or more integrated circuit (IC) chips. For example, in oneimplementation, a set of components can be implemented in a single ICchip. In other implementations, one or more of respective components arefabricated or implemented on separate IC chips.

What has been described above includes examples of the implementationsof the present invention. It is, of course, not possible to describeevery conceivable combination of components or methodologies forpurposes of describing the claimed subject matter, but it is to beappreciated that many further combinations and permutations of thisinnovation are possible. Accordingly, the claimed subject matter isintended to embrace all such alterations, modifications, and variationsthat fall within the spirit and scope of the appended claims. Moreover,the above description of illustrated implementations of this disclosure,including what is described in the Abstract, is not intended to beexhaustive or to limit the disclosed implementations to the preciseforms disclosed. While specific implementations and examples aredescribed herein for illustrative purposes, various modifications arepossible that are considered within the scope of such implementationsand examples, as those skilled in the relevant art can recognize.

In particular and in regard to the various functions performed by theabove described components, devices, circuits, systems and the like, theterms used to describe such components are intended to correspond,unless otherwise indicated, to any component which performs thespecified function of the described component (e.g., a functionalequivalent), even though not structurally equivalent to the disclosedstructure, which performs the function in the illustrated exemplaryaspects of the claimed subject matter (e.g., media item analysis,synchronization, and aggregation). In this regard, it will also berecognized that the innovation includes a system as well as acomputer-readable storage medium having computer-executable instructionsfor performing the acts and/or events of the various methods of theclaimed subject matter.

The aforementioned systems/circuits/modules have been described withrespect to interaction between several components/blocks. It can beappreciated that such systems/circuits and components/blocks can includethose components or specified sub-components, some of the specifiedcomponents or sub-components, and/or additional components, andaccording to various permutations and combinations of the foregoing.Sub-components can also be implemented as components communicativelycoupled to other components rather than included within parentcomponents (hierarchical). Additionally, it should be noted that one ormore components may be combined into a single component providingaggregate functionality or divided into several separate sub-components,and any one or more middle layers, such as a management layer, may beprovided to communicatively couple to such sub-components in order toprovide integrated functionality. Any components described herein mayalso interact with one or more other components not specificallydescribed herein but known by those of skill in the art.

Notwithstanding that the numerical ranges and parameters setting forththe broad scope of the invention are approximations, the numericalvalues set forth in the specific examples are reported as precisely aspossible. Any numerical value, however, inherently contains certainerrors necessarily resulting from the standard deviation found in theirrespective testing measurements. Moreover, all ranges disclosed hereinare to be understood to encompass any and all sub-ranges subsumedtherein. For example, a range of “less than or equal to 10” can includeany and all sub-ranges between (and including) the minimum value of zeroand the maximum value of 10, that is, any and all sub-ranges having aminimum value of equal to or greater than zero and a maximum value ofequal to or less than 10, e.g., 1 to 5. In certain cases, the numericalvalues as stated for the parameter can take on negative values. Inaddition, while a particular feature of this innovation may have beendisclosed with respect to only one of several implementations, suchfeature may be combined with one or more other features of the otherimplementations as may be desired and advantageous for any given orparticular application. Furthermore, to the extent that the terms“includes,” “including,” “has,” “contains,” variants thereof, and othersimilar words are used in either the detailed description or the claims,these terms are intended to be inclusive in a manner similar to the term“comprising” as an open transition word without precluding anyadditional or other elements.

Reference throughout this specification to “one implementation” or “animplementation” or “one embodiment” or “an embodiment” means that aparticular feature, structure, or characteristic described in connectionwith the implementation is included in at least one implementation or atleast one embodiment. Thus, the appearances of the phrase “in oneimplementation” or “in an implementation” or “in one embodiment” or “inan embodiment” in various places throughout this specification are notnecessarily all referring to the same implementation/embodiment.Furthermore, the particular features, structures, or characteristics maybe combined in any suitable manner in one or moreimplementations/embodiments.

Further, references throughout this specification to an “item,” or“file,” means that a particular structure, feature or object describedin connection with the implementations are not necessarily referring tothe same object. Furthermore, a “file” or “item” can refer to an objectof various formats. While referees to media items generally refer tovideo items (a series of image files with or without audio), it is to beappreciated that media items may be of various formats.

As used in this application, the terms “component,” “module,” “system,”or the like are generally intended to refer to a computer-relatedentity, either hardware (e.g., a circuit), a combination of hardware andsoftware, or an entity related to an operational machine with one ormore specific functionalities. For example, a component may be, but isnot limited to being, a process running on a processor (e.g., digitalsignal processor), a processor, an object, an executable, a thread ofexecution, a program, and/or a computer. By way of illustration, both anapplication running on a controller and the controller can be acomponent. One or more components may reside within a process and/orthread of execution and a component may be localized on one computerand/or distributed between two or more computers. While separatecomponents are depicted in various implementations, it is to beappreciated that the components may be represented in one or more commoncomponent. Further, design of the various implementations can includedifferent component placements, component selections, etc., to achievean optimal performance. Further, a “device” can come in the form ofspecially designed hardware; generalized hardware made specialized bythe execution of software thereon that enables the hardware to performspecific function (e.g., media item aggregation); software stored on acomputer readable medium; or a combination thereof.

Moreover, the words “example” or “exemplary” are used herein to meanserving as an example, instance, or illustration. Any aspect or designdescribed herein as “exemplary” is not necessarily to be construed aspreferred or advantageous over other aspects or designs. Rather, use ofthe words “example” or “exemplary” is intended to present concepts in aconcrete fashion. As used in this application, the term “or” is intendedto mean an inclusive “or” rather than an exclusive “or”. That is, unlessspecified otherwise, or clear from context, “X employs A or B” isintended to mean any of the natural inclusive permutations. That is, ifX employs A; X employs B; or X employs both A and B, then “X employs Aor B” is satisfied under any of the foregoing instances. In addition,the articles “a” and “an” as used in this application and the appendedclaims should generally be construed to mean “one or more” unlessspecified otherwise or clear from context to be directed to a singularform.

What is claimed is:
 1. A media system, comprising: a memory; and aprocessor that executes computer-executable instructions stored in thememory that cause the processor to: receive first metadata associatedwith a first video item captured by a first user device and secondmetadata associated with a second video item captured by a second userdevice; determine, based at least on the first metadata and the secondmetadata, that the first video item and the second video item areassociated with each other; determine a timeline such that a first timein the first video item is synchronized with a second time in the secondvideo item; aggregate the first video item and the second video iteminto a composite video item that includes at least a first segment fromthe first video item and a second segment from the second video item,wherein the first segment and the second segment are arranged in thecomposite video in synchronization with the determined timeline; causethe composite video item to be presented on a third user device in whichthe first video item and the second video item are simultaneously playedback based on the synchronization, wherein the composite video item is amosaic in which the first video item and the second video item areconcurrently displayed; and modify presentation of the composite videoitem at a third time on the third user device by modifying a visualcharacteristic with which the first video item is presented within themosaic based on a user preference indicated by a user of the first userdevice.
 2. The system of claim 1, wherein the processor is furtherconfigured to receive first audio data corresponding to the first videoitem and second audio data corresponding to the second video item,wherein the composite video includes a combination of the first audiodata at a first loudness level and the second audio data at a secondloudness level, and wherein the first loudness level and the secondloudness level are indicated by the user preference.
 3. The system ofclaim 1, wherein the processor is further configured to rank respectivesegments of the first video item and the second video item as a functionof at least one of a frequency of appearance of an entity in therespective segments, respective viewing angles of an object captured inthe respective segments, respective video qualities of the respectivesegments, or respective audio qualities of the respective segments, andwherein the respective segments include the first segment and the secondsegment.
 4. The system of claim 1, wherein the processor is furtherconfigured to rank respective segments of the first video item and thesecond video item based at least in part on the user preference.
 5. Thesystem of claim 1, wherein the processor is further configured to modifythe aggregation of the first video item and the second video item basedat least in part on the user preference.
 6. The system of claim 1,wherein the processor is further configured to: receive a request duringthe presentation of the composite video that includes a request to panthe composite video item; and cause the composite video item to switchfrom presenting the first video item to the second video item based on afirst field of view associated with the first video item and a secondfield of view associated with the second video item.
 7. The system ofclaim 1, wherein the composite media item is caused to be presented onthe third user device via at least one of broadcasting, streaming, oruploading of the composite media item.
 8. The system of claim 1, whereinmodifying the visual characteristic with which the first video item ispresented comprises modifying a focus of the first video item.
 9. Amethod, comprising: receiving, at a server, first metadata associatedwith a first video item captured by a first user device and secondmetadata associated with a second video item captured by a second userdevice; determining, by the server, based at least on the first metadataand the second metadata, that the first video item and the second videoitem are associated with each other; determining, by the server, atimeline such that a first time in the first video item is synchronizedwith a second time in the second video item; aggregating, by the server,the first video item and the second video item into a composite videoitem that includes at least a first segment from the first video itemand a second segment from the second video item, wherein the firstsegment and the second segment are arranged in the composite video insynchronization with the determined timeline; causing, by the server,the composite video item to be presented on a third user device in whichthe first video item and the second video item are simultaneously playedback based on the synchronization, wherein the composite video item is amosaic in which the first video item and the second video item areconcurrently displayed; and modifying presentation of the compositevideo item at a third time on the third user device by modifying avisual characteristic with which the first video item is presentedwithin the mosaic based on a user preference indicated by a user of thefirst user device.
 10. The method of claim 9, further comprisingreceiving first audio data corresponding to the first video item andsecond audio data corresponding to the second video item, wherein thecomposite video includes a combination of the first audio data at afirst loudness level and the second audio data at a second loudnesslevel, and wherein the first loudness level and the second loudnesslevel are indicated by the user preference.
 11. The method of claim 9,further comprising ranking respective segments of the first video itemand the second video item as a function of at least one of a frequencyof appearance of an entity in the respective segments, respectiveviewing angles of an object captured in the respective segments,respective video qualities of the respective segments, or respectiveaudio qualities of the respective segments, and wherein the respectivesegments include the first segment and the second segment.
 12. Themethod of claim 9, further comprising ranking respective segments of thefirst video item and the second video item based at least in part on theuser preference.
 13. The method of claim 9, further comprising modifyingthe aggregation of the first video item and the second video item basedat least in part on the user preference.
 14. The method of claim 9,further comprising: receive a request during the presentation of thecomposite video includes a request to pan the composite video item; andcausing the composite video item to switch from presenting the firstvideo item to the second video item based on a first field of viewassociated with the first video item and a second field of viewassociated with the second video item.
 15. The method of claim 9,wherein the composite media item is caused to be presented on the thirduser device via at least one of broadcasting, streaming, or uploading ofthe composite media item.
 16. The method of claim 9, wherein modifyingthe visual characteristic with which the first video item is presentedcomprises modifying a focus of the first video item.
 17. Anon-transitory computer-readable medium comprising computer-executableinstructions that, when executed by a processor, cause the processor toperform a method, the method comprising: receiving first metadataassociated with a first video item captured by a first user device andsecond metadata associated with a second video item captured by a seconduser device; determining, based at least on the first metadata and thesecond metadata, that the first video item and the second video item areassociated with each other; determining a timeline such that a firsttime in the first video item is synchronized with a second time in thesecond video item; aggregating the first video item and the second videoitem into a composite video item that includes at least a first segmentfrom the first video item and a second segment from the second videoitem, wherein the first segment and the second segment are arranged inthe composite video in synchronization with the determined timeline;causing the composite video item to be presented on a third user devicein which the first video item and the second video item aresimultaneously played back based on the synchronization, wherein thecomposite video item is a mosaic in which the first video item and thesecond video item are concurrently displayed; and modifying presentationof the composite video item at a third time on the third user device bymodifying a visual characteristic with which the first video item ispresented within the mosaic based on a user preference indicated by auser of the first user device.